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EXECUTIVE SUMMARY 


This report is based on data obtained from the 2018 National Survey of Postsecondary Competency-Based 


Education (NSPCBE). It represents a collaborative effort between the American Institutes for Research (AIR) 
and Eduventures®, the research division of ACT™ | NRCCUA™. It includes findings based on responses by more 
than 500 colleges and universities—the largest institutional sample to date—about their interest in, or 
activity related to, competency-based education (CBE). It also compares findings to those of Eduventures’ 
2016 CBE study, Deconstructing CBE. 


The goal of the 2018 NSPCBE is to provide a comprehensive and detailed assessment of the state of 

postsecondary CBE. It includes data and findings that are relevant to a range of institutional stakeholders, 
policymakers, researchers, and educators. It includes six key findings, based on 2018 data, and, wherever 
possible, comparisons to the 2016 data from prior Eduventures’ research. These findings include the following: 


1. Motivations for adopting CBE: Institutions see CBE as a way to serve nontraditional students and 
improve workforce readiness. 


2. Scope of CBE adoption: Many institutions’ adoption activities fall short of full CBE programming. 


3. Scale of enrollment in CBE programs: Most CBE programs currently serve relatively small numbers 
of students. 


4. Role of faculty in CBE programs: Faculty are still fulfilling a broad range of roles in active CBE programs. 


5. Barriers to CBE implementation: Perceived barriers to CBE implementation represent both internal and 
external factors. 


6. Future of CBE: Most institutions are optimistic about the future of CBE. 


Each finding includes a detailed analysis of the available data and explores the implications for both 
schools with active CBE programs as well as those in the planning stage. Collectively, these findings 
suggest that although CBE remains a compelling and valuable innovation, further growth and scalable 
impact are hampered by a range of barriers. These barriers stem from both the federal policies and the 
external regulatory climate as well as those that are internal to the operations and governance of many 
institutions of higher education. 


This 2018 NSPCBE report concludes with critical questions for postsecondary CBE implementation and 
scale, followed by a set of recommendations for program leaders, institutional leaders, and policymakers. 


Future editions of the NSPCBE, currently planned for 2019 and 2020, will continue to address these 
questions and track the evolution of the field over time. 
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INTRODUCTION 


Study Background and Rationale 


Competency-based education (CBE) is a nontraditional, but 
not necessarily new, approach to postsecondary education. 
In the last decade, it has attracted considerable attention 
from a wide variety of stakeholders, policymakers, and 
institution leaders. Although exact definitions of CBE vary, 
several components distinguish CBE from traditional models 
of postsecondary education: 


@ Curricula are designed around specific competencies, 
™ Advancement focuses on demonstration of competency, and 


™ The time it takes to demonstrate a competency is typically 


allowed to vary. 


Proponents of CBE argue that its “learner-centered” logic is compelling: By measuring students’ learning 
rather than a traditional program’s credit hours and grades, CBE has the potential to improve quality of 
learning, expand access for “nontraditional” students,* and lower costs for students. 


But compelling logic alone is insufficient: Institutional leaders interested in CBE need to know whether their 
peers are adopting it, how, and what the path ahead might look like. Similarly, policymakers want to know 
whether CBE is expanding across institution types and what barriers are inhibiting growth as they consider 
whether and how to design policies that support or inhibit CBE expansion. 


This report summarizes findings from a survey of college and university leaders that sought to better 

understand whether, how, when, why, and for whom institutions in the United States are using CBE. This 
survey, a partnership of the American Institutes for Research (AIR) and Eduventures, a division of ACT | 
NRCCUA,? was conducted in 2018. It builds upon and updates Eduventures’ Deconstructing CBE report, 
a study of 251 institutions supported by Ellucian.* The 2018 NSPCBE begins a 3-year study, supported 
by Lumina Foundation, to better understand CBE scale and adoption. Together with Eduventures' 2016-17 
research, it provides an essential baseline for further inquiry into the future of postsecondary CBE. 


In 2016, Eduventures found that although interest in CBE was high, outside of a few large and established 
schools where CBE was the dominant mode of instruction, implementation remained relatively fragmented, 
small in scale, and generally designed to meet the needs of working adult learners. Since 2016, a variety 
of factors have affected both expectations for broader CBE growth as well as the pace and scale of 
actual implementation. These factors include improved knowledge about CBE through hubs such as the 


1 This report uses “nontraditional” students to refer broadly to student populations that are older than students coming directly from high school (typically age 25 or older), in line 
with the description provided by the National Center for Education Statistics: https://nces.ed.gov/ pubs/web/97578e.asp. These populations are referred to as “new traditional” 
or “today’s students” as well. 

2 The National Research Center for College and University Admissions 

3 Garrett, R., & Lurie, H. (2016). Deconstructing CBE: An assessment of institutional activity, goals, and challenges in higher education. Boston, MA: Eduventures. 
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Competency-Based Education Network (C-BEN), the continued growth of prominent CBE exemplars, more 
specific guidance from accreditors, and inconsistent messages from the federal government regarding the 
use of Title IV funding in certain CBE models.* Given the evolving nature of the field—and to begin tracking 
developments in the field longitudinally—the 2018 NSPCBE reexamines the state of postsecondary CBE 
with the most comprehensive study to date about implementation of CBE. 


Broadly, this survey shows that although the learner-centric logic of CBE remains compelling across 
institution types and that more than 500 programs have been launched and are operating, significant 
barriers to implementation and scale remain. These barriers are often related to core functions of 
the higher education enterprise and lead many institutions to adopt CBE on a relatively small scale 
or to engage in partial efforts to implement programs. There is, however, evidence that a majority of 
respondents believe CBE will grow nationally. 


This report highlights six key findings: 


1. Motivations for adopting CBE: Institutions see CBE as a way to serve nontraditional students and 
improve workforce readiness. 


2. Scope of CBE adoption: Many institutions’ adoption activities fall short of full CBE programming. 


3. Scale of enrollment in CBE programs: Most CBE programs currently serve relatively small numbers 
of students. 


4. Role of faculty in CBE programs: Faculty are still fulfilling a broad range of roles in active CBE programs. 


5. Barriers to CBE implementation: Perceived barriers to CBE implementation represent both internal and 
external factors. 


6. Future of CBE: Most institutions are optimistic about the future of CBE. 


Appendix A includes descriptive statistics about responses to key survey questions, many of which shed 
further light on patterns of implementation among those who are implementing CBE. It is important to 
note that this study is based on a survey of institutions, so findings are based on self-reported data 
from institution leaders. Although this survey does not include an evaluation of student outcomes and 
does not incorporate the perspectives of students, employers, or other stakeholders, we recognize the 
importance of that information and encourage continued research in those areas. 


* Signals from the federal government include a positive emphasis on CBE in legislative proposals, such as the PROSPER Act, as well as the 2017 findings from regulatory 
enforcement regarding Western Governors University, a well-known CBE model. 
U.S. House of Representatives Committee on Education and the Workforce. (2017). PROSPER Act: Promoting Real Opportunity, Success, and Prosperity through Education Reform 
(PROSPER) Act. Washington, DC. Retrieved from https://edworkforce.house.gov/prosper/ 
U.S. Department of Education, Office of Inspector General. (2017). Western Governors University was not eligible to participate in the Title IV programs: Final audit report. 
Washington, DC. Retrieved from https://www2.ed.gov/about/ offices/list/ oig/auditreports/fy2017/a05m0009. pdf 
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Methodology 


SURVEY DEVELOPMENT 


To best capture longitudinal data while collecting new relevant information, AIR and Eduventures designed 
the 2018 NSPCBE survey instrument to build on Eduventures’ 2016 Deconstructing CBE survey. This 
approach was guided and vetted by an advisory board of key leaders and experts involved in leading or 
studying CBE; the advisory board provided insight about what questions to maintain and what questions 
might need to change based on developments in the field.° Many questions on the 2018 survey instrument 
are consistent with those on the 2016 instrument—particularly those related to the elements associated 
with CBE, the path toward implementation, and details about implementation. Changes in the 2018 
instrument were primarily to solicit additional details, add clarity or updated terminology to existing 
questions, or remove questions from 2016 that did not yield much variation.® 


SAMPLE 


The 2018 NSPCBE was sent to 3,043 institutions, representing a census of 2- and 4-year institutions listed 
in the Integrated Postsecondary Education Data System (IPEDS).’ For most recipients, the survey invitation 
went to provosts and institutional research contacts with a request that the recipient forward it to the most 
appropriate contact on campus, which may vary by campus. C-BEN member institutions (as of 2017-18) 
were an exception; they provided preferred contact information for their institutions directly in advance. 


Of the 3,043 institutions, 501 provided a response, representing the largest survey to date of institutions 

about CBE and an overall institutional response rate of 16%. Of that 501, 54% were from public institutions, 
41% were from private nonprofit institutions, and 5% were from private for-profit institutions (See Figure 1). 
The majority—69%—were from 4-year institutions, with 31% from 2-year institutions (See Figure 2). 


Figure 1. Respondents by Sector Figure 2. Respondents by Level 
100% 100% 
80% 80% 
60% 60% 
40% 40% 
20% 20% 
0% 0% 


|| Public || Private nonprofit ial Private for-profit ea 4-year || 2-year 


5 Fora list of board members, please see the AIR CBE Research website: https://cberesearch.org/about-us#board 

5 For example, edits improved wording, narrowed response options related to types of students served, and eliminated questions about implementation details (including questions 
about how technology is used and details about roles; Eduventures and AIR determined that those are valuable but more feasible for interview approaches). 

7 Recipient institutions represented those in the IPEDS universe for which we could obtain contact information from the Higher Education Directory. For more information about the 
sample, see Appendix B. 
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The survey asked respondents to identify their Figure 3. Respondents’ Roles on Campus 


role to clarify their perspective on CBE. Thirty-four 

percent identified as a chief academic affairs 

officer (provost or vice president of learning), 30% 

identified as the institutional research officer, and 

11% identified as a dean. The remaining 25% of 

respondents identified as presidents/chancellors, Sa 


vice provost/provost’s office staff, department 
chairs, faculty members, or other (see Figure 3). 


Eighty-one percent of respondents indicated 
that they had “institutionwide” knowledge about 
competency-based approaches compared with 
11% who noted that their scope of knowledge 
was limited to a single academic unit. 


Institutional Research 
Officer 


Chief Academic 
Affairs Officer 


Because institutions that were adopting or il meen a nee 

interested in adopting CBE may be more likely 

to respond than those without interest, sample bias is possible. To mitigate this possible sample bias, 
weights were assigned to each survey respondent based on how likely comparable institutions were to 
respond to the survey. Although these weights do not affect counts (including the number of institutions 
and programs listed throughout this Methodology section), they are used to calculate percentages in the 
Key Findings section of this report and its appendices. For a more detailed description of the survey 
weights and overall methodological approach, see Appendix B. 


DEFINING CBE 


To define CBE while acknowledging variation within CBE programs, survey respondents were asked to 
answer a series of questions regarding their adoption of or interest in several elements associated with 
competency-based approaches.® Next, the survey asked those who had adopted those elements whether 
they had adopted those approaches at the course level or for entire programs of study. 


For the purposes of this study, the threshold for classification as a CBE program was whether an entire 
program contained at least one of the following characteristics: 


1. Learning is measured in competencies, and either quantified without reference to seat time or mapped 
to measures of seat time; 


2. Students advance from the course or complete the program based on mastering all required 
competencies; or 


3. Courses or programs can be substantially “self-paced”? by students. 


8 Respondents were grouped into categories based on their adoption and interest, including (1) adopted or adoption in-progress, (2) interested but not adopting, and (3) not 
interested in CBE. 

° The authors acknowledge that “self-paced” can be defined differently by different respondents, and that some in the CBE field use different terminology, such as “flexible” or 
“personalized” pacing. For the purposes of this survey, “self-paced” was meant to encompass any situation where expectations for learning are held constant while time is allowed to 
vary to some degree. 
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These criteria were selected with input from the NSPCBE advisory board, which sought to balance 
(1) accounting for the key components of widely recognized definitions in the field,*° and (2) capturing 
the variety of program types that exist at this point in the evolution of CBE.** 


The survey also sought to capture information about institutions that were implementing competency- 
based elements, but that fell short of the CBE definition threshold. This approach allows for analysis 
of adoption that falls short of a full CBE program, but may signal future activity. The prevalence of all 
elements, as well as activity at the course level rather than the program level, is discussed in the Key 
Findings section of this report. 


This survey does not, however, attempt to include the full set of related approaches, termed “competency- 
based learning” (CBL) approaches. CBL may include structured and unstructured opportunities for learning 
and/or the assessment of learning, both self-created and those designed by employers, education institutions, 
and training providers, which are aligned to competencies and may lead to a recognized education credential. 
These approaches may include military training, apprenticeships and workforce development programs, and 
other related opportunities.*? 


KEY INDICATORS 


STATE OF THE FIELD 


Four hundred thirty of the 501 respondents reported being either interested in adopting CBE or in the 
process of adopting; 71 expressed no interest in CBE. Fifty-seven institutions reported currently operating 
at least one full CBE program; together, those 57 institutions reported offering a total of 512 CBE programs: 
427 undergraduate programs and 85 graduate programs. The highest concentration of programs reported 
were in nursing and computer sciences. 


10 See, for example, the Competency-Based Education Network's definition of CBE: https://www.cbenetwork.org/competency-based-education/ 
"| For example, this threshold allows analyses to include implementation of both direct assessment and nondirect assessment CBE programs. 
12 For a more complete description of the distinction between CBL and CBE, see https://cberesearch.org/about-us 
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KEY FINDINGS 


This report features six key findings from the 2018 NSPCBE. 
Descriptive statistics from other sections of the survey also 
are included in Appendix A. 


1. Motivations for Adopting CBE 


Key finding: Institutions see CBE as a way to serve 


nontraditional students and improve workforce readiness. 


To better understand which aspects of CBE were most 
appealing to institutions that were adopting CBE or 
interested in adoption, the NSPCBE asked respondents 
to identify the top motivations behind program adoption 
or interest. 


For institutions with existing CBE programs and those that 
have begun adopting programs, the rationale for adopting CBE focused on expanding access to certificate 
or degree programs and preparing students for the workforce (see Figure 4): 


M@ Fifty-five percent reported that they considered CBE as a means to expand opportunities for nontraditional 
students, and 


M™ Fifty-six percent seek to better prepare their students for the workforce. 


Figure 4. Motivation for CBE: Institutions With a Program or Currently Adopting 


Part of effort to improve learning outcomes 

Response to workforce needs 

Desire to clarify learning outcomes 

Desire to lower tuition for non-traditional learners 

Part of a broader initiative on education innovation 
Desire to lower tuition for certain programs 

Desire to enhance student employability 

Desire to improve completion rates 

Desire to expand opportunities for nontraditional learners 
Desire to increase enrollment 


Other 


0% 20% 40% 60% 80% 100% 


BM 4-year 2-year Total 
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Among institutions with only interest (but no program to date) in CBE, expanding opportunities for 
nontraditional students and improving learning outcomes were among the top responses (see Figure 5): 


® Sixty-three percent reported that CBE's potential to expand opportunities for nontraditional students 
was a key factor, and 


H Fifty-five percent reported that interest in CBE was part of an effort to improve learning outcomes 
for students. 


Figure 5. Motivation for CBE: Institutions With Interest in Adopting 
Part of effort to improve learning outcomes 

Response to workforce needs 

Desire to clarify learning outcomes 

Desire to lower tuition for non-traditional learners 

Part of a broader initiative on education innovation 
Desire to lower tuition for certain programs 

Desire to enhance student employability 

Desire to improve completion rates 

Desire to expand opportunities for nontraditional learners 
Desire to increase enrollment 


Other 


0% 20% 40% 60% 80% 100% 


BM 4year 2-year Total 


Boosting overall enrollment was not a top priority for either group of institutions; the share reporting this 
as a goal was at or below 40% for both groups. 


Although these priorities are largely consistent between 2-year and 4-year institutions, there are some 
notable differences. The focus on workforce preparation is more pronounced for 2-year institutions, likely 
reflecting their traditional mission of providing workplace-focused credentials.*% 


Another key difference rests in a focus on completion: A greater share of 2-year institutions reported valuing 
CBE for its potential to improve completion rates compared with their 4-year counterparts. This reflects the 
priority that many 2-year institutions have related to improving completion rates, and suggests that many 
may see CBE as a model that could help boost completion. 


13 This difference is statistically significant. 
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In addition to comparing 2- and 4-year institutions, these trends were consistent across sector and 
institutional size. 


Comparison to 2016: These themes are consistent with 2016 findings; the focus on access for nontraditional 
learners and workforce relevance were among the top reasons cited in 2016, with improving completion rates 

near the top of the list as well. The desire to lower tuition was similarly the least common rationale in 2016. 

The consistent focus on supporting nontraditional students also may reflect increasing interest among more 

schools to improve their programming and support systems for working adults. 


2.Scope of CBE Adoption 


Key finding: Many institutions' adoption activities fall short of full CBE programming. 


Two key takeaways from Eduventures’ 2016 report were that (1) CBE is not a well-defined model, and many 
institutions were using a variety of elements commonly associated with CBE even if they were not fully 
adopting a program; and (2) most CBE activity remained aspirational, still at the planning or course-level 
implementation stage. 


The 2018 NSPCBE was designed to explore how the CBE landscape may have evolved since 2016 and 
whether or not more institutions had moved to full programs with key components of CBE in place. To 
that end, respondents were asked about activity that may not meet the definition of a full CBE program, 
but may be close on two dimensions: (1) implementation of specific elements related to CBE and (2) CBE 
implementation at the course level or still in the planning phase. Overall, the trend is still similar to 2016: 
Much of the activity remains short of a full CBE program on at least one of those two dimensions. This 
section describes the activity underlying that overarching trend. 


ADOPTION BY ELEMENTS 


Just over half of institutions reported having adopted—or planning to adopt—at least one of the elements 
that would meet the minimum threshold for CBE in this survey: 


1. Measuring learning in competencies, either quantified without reference to seat time or mapped to 
measures of seat time; 


2. Requiring mastery of all required competencies for advancement between unit to unit or for program 
completion; and 


3. Allowing students to substantially “self-pace” (or “personalize the pace” of) courses or programs. 


Each of those individual elements was adopted by less than half of respondents, though, indicating that 
many institutions are not implementing all of the above (see Figure 6); rather, they appear to be selectively 
implementing certain elements. 


In contrast, elements related to CBE but not independently meeting the threshold for definition as CBE 
were typically more prevalent. These elements are often associated with CBE or may represent early steps 
toward adoption. These elements include: 


1. Writing clear definitions of competencies at both the course and program levels (an activity often 
related to accreditation); 
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2. Using prior learning assessments (PLA) for both awarding credit and for placement or personalization 
purposes for incoming students; and 


3. Codeveloping competencies with employers. 
Figure 6. Adoption Activity by Element (Adopted or In Progress) 


. 2-year 
Substantially self-paced by students 
4-year 


Program completion based on 2-year 
mastery of all competencies 4-year 


Course to course advancement based 2-year 
on mastering all competencies 4-year 


Learning measured in competencies 2-year 
mapped to measures of seat time 4-year 


Elements meet threshold for CBE. 


Learning measured in competencies 2 Year 
without reference to seat time 4-year 


Competencies are co-developed 2-Year 
with employers 4-year 


; _ 2-year 
PLA used for awarding credit 
4-year 


PLA used for placement 2-year 
or personalization 4-year 


Clear definitions of competencies, 2-year 
program level 4-year 


Elements do not meet threshold for CBE. 


Clear definitions of competencies, 2-Yea" 


course level 4-year 


0% 20% 40% 60% 80% 100% 


The only element that had a statistically significant difference in adoption between 2-year and 4-year 
institutions was the codevelopment of competencies with employers or other third parties. Two-year 
institutions reported significantly higher adoption than 4-year institutions, potentially reflecting the 
traditional distinctions in employer engagement between the two segments. 


Comparison to 2016: Eduventures’ 2016 survey showed that institutions were selectively adopting certain 
elements associated with CBE. In particular, in 2016, defining learning outcomes at the course and program 
levels were the most commonly adopted elements, followed by options related to using PLAs.** One difference 


4 Elements included in the 2016 and 2018 surveys differed slightly. The 2016 survey did not ask about mastery as the basis for advancement in this section. The 2018 survey 
did not include adaptive learning or learning outcomes at the subcourse level, and it collapsed three PLA options (placement, personalization, and credit) into two (placement/ 
personalization and credit). 
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is that the use of direct assessment (measuring learning and quantifying it without reference to a measure 
of seat time) was among the most commonly reported adopted or adoption-in-progress elements in 2016; 
in 2018, however, it was one of the least commonly reported elements. This difference may be due, in 
part, to the different set of institutions responding to the 2018 survey. 


STAGE OF ADOPTION 


The 2018 NSPCBE also asked institutions that indicated current or in-process adoption of elements that 
met the threshold of CBE to clarify their current stage of implementation. It sought to distinguish whether 
institutions were still at the planning stage (“adoption in progress”), implementing elements in individual 
courses, or implementing full programs. As shown in Figure 7, the majority—57%—report that they were 
still in the planning stage. Thirty-two percent reported activity at the course level (one course or multiple), 
while only 11% reported having one or more full programs. Of those with at least one full program, having 
two to five programs was the most common category—with fewer institutions reporting that CBE was the 
“predominant” approach at their institution. 


Figure 7. Stage of Adoption 


100% 
80% 
60% 
40% 
20% 
0% 

Planning One Multiple Multiple One Two to five More than Institution is 

stage course courses in courses across program programs five programs — predominantly 

vone program multiple programs CBE 
Pe ee N__ uN! 
Course-level adoption Program-level adoption 


BN Share of respondents 


Comparison to 2016: These findings are consistent with the overall results from Eduventures’ 2016 study, 
which found that 38% were at the planning stage and 37% were only active at the course level. Although 
these broad trends are not longitudinal data about the same institutions and how they have changed over 
time, they generally show the prevalence of course-level activity rather than full program(s) in both 2016 
and 2018 as well as the concentration of institutions still in the planning phase. 
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3.Scale of Enrollment in CBE Programs 


Key finding: Most CBE programs currently serve relatively small numbers of students. 


For those institutions offering full CBE programs, a key indicator of scale and potential impact of implementation 
is student enrollment. To better understand the scale of enrollment, the survey asked institutions with existing 
CBE programs to provide recent estimates of student enrollment. 


Institutions with undergraduate CBE programs reported relatively small enrollments in those programs. 
In the last academic year, 53% of those reporting undergraduate enrollment in CBE programs reported 

fewer than 50 students per program (see Figure 8). In comparison, just 4.0% enrolled more than 1,000 
undergraduate students in CBE programs. When disaggregating findings by institution level, there is a 

somewhat similar pattern, with approximately half of programs at both the 2- and 4-year levels enrolling 

fewer than 50 students. 


Figure 8. Reported Undergraduate Enrollment in Active CBE Programs: 
Share of Institutions in Enrollment Size Categories 


100% 
80% 
60% 
40% 
20% 


0% 
0-50 51-100 101-200 201-499 500-1000 >1000 Not sure 
Undergraduate Enrollment 


BM 4vear I 2-year Total 


The prevalence of small programs in the 2018 survey suggests that, except for a small set of institutions 
with large programs, CBE at most institutions has not yet achieved large-scale implementation. Some of the 
reported barriers to adoption, detailed in the finding summarizing barriers, may explain this phenomenon, or 
this finding may represent early stage programs that institutions are piloting at a small scale. 


Comparison to 2016: Although data limitations*® make comparisons with 2016 challenging, data collected 
in 2016 show a similar concentration of programs with enrollment below 100 students. This was consistent 
among both undergraduate and graduate programs reported in the 2016 survey. 


1 This finding is difficult to compare because a relatively small share of respondents have operating programs and report enrollment sizes, so estimates are based on smaller sample 
sizes, and the enrollment size ranges were slightly different. 
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4.Role of Faculty in CBE Programs 


Key finding: Faculty are still fulfilling a broad range of roles in active CBE programs. 


Because CBE often departs from traditional models of teaching and learning, CBE programs frequently 
involve restructuring typical faculty roles.*® The identification of optimal ways to structure faculty roles for 
successfully supporting student learning has been a key implementation challenge in many CBE programs. 
In addition, monitoring contact between faculty and students continues to be a core focus of the Education 
Department through the existing "regular and substantive" regulation.*” To better understand what faculty 
roles look like in 2018, the NSPCBE included questions about the types of roles that faculty and staff filled 
in operating CBE programs. 


Generally, responses support the notion that faculty continue to fulfill a broad range of roles across CBE 
programs, with the majority reporting that faculty members provide direct instruction to students, evaluate 
student performance on assessments, define competencies, develop assessments, and design instructional 
content (see Figure 9). Roles less commonly fulfilled by faculty include participating in program review, training 
other faculty or staff, engaging with third parties to identify competencies, coaching/mentoring students, and 
interacting with support teams; institutions typically reported that these roles were fulfilled by nonfaculty staff. 
These findings suggest that faculty in CBE programs fulfill content-driven roles but are likely to rely upon the 
contributions of nonfaculty staff to provide advising and related student support services. 


Figure 9. Faculty Roles in CBE 
Provide direct instruction to students 
Evaluate student performance on assessments 
Generate/ refine competencies 
Develop assessments 
Design instructional content 
Participate in program review and improvement 
Train other faculty or staff interested in CBE 


Coach students on their academic performance 


Engage with business and industry 
to identify relevant competencies 


Mentor students about career options 
Interact with support teams 


Advise students about course or program requirements 


0% 20% 40% 60% 80% 100% 
WM Facutty = Both faculty and staff 


16 Navarre Cleary, M. (2015). Faculty and staff roles and responsibilities in the design and delivery of competency-based education programs: A C-BEN snapshot. Franklin, TN: 
Competency-Based Education Network. Retrieved from http://works.bepress.com/navarrecleary/14/ 


17 Mehaffie, L. B. (2014, December 19). Competency-based education programs—Questions and answers. Washington, DC: U.S. Department of Education, Federal Student Aid. 
Retrieved from https://ifap.ed.gov/dpcletters/GEN1423.html 
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Comparison to 2016: The 2018 findings are broadly 
consistent with 2016, with a few notable exceptions. Areas 
of consistency include the development and evaluation of 
assessments and the creation of instructional content. In 
2018, however, institutions more commonly reported that 
faculty were involved in additional areas, including direct 
instruction activities, developing and refining competencies, 
and training other faculty or staff (though, generally, this 
was much less common than the first two roles). Faculty 
involvement in other areas, including program evaluation and 
interaction with student support teams, was generally less 
common than in 2016. Most notable is the lower prevalence 
of institutions reporting that faculty are directly involved in 
coaching, mentoring, and advising students on a range of 
topics; this finding may be due in part to the evolving ways 
in which CBE programs are framing and structuring this role or splitting some elements between faculty and 
staff roles.*® 


It is clear from both the 2016 and 2018 surveys that CBE programs use faculty in a broad range of roles. A 
key consideration, though, is that responses do not distinguish how and whether programs have arranged 
“unbundled” or “reassembled” faculty models, which would involve having specific faculty fulfill one or 
two roles while other faculty members specialize in another role (e.g., instructional faculty separate from 
faculty evaluating performance on assessments).*? Institutions may be reporting that faculty fulfill many 
roles, but it may be that different faculty fulfill different types of roles. Given the potential importance of 
this finding, questions about this approach will be included in future years of the survey. 


5. Barriers to CBE Implementation 


Key finding: Perceived barriers to CBE implementation represent both internal and external factors. 


Because CBE may be a less familiar approach to teaching and learning, program leaders may encounter 
local barriers to implementation on their campuses, ranging from institutional processes and infrastructure 
to stakeholder buy-in. External factors, such as accreditation and financial aid regulations, also may affect 

an institution’s decision to implement CBE. Given the broad range of potential barriers, the NSPCBE asked 
respondents to share their perceptions of what barriers exist to implementing CBE on their campuses. 


Across institution types and levels of program adoption, responses indicated several key barriers to CBE 
implementation. More than 50% of institutions responding cited three key barriers, regardless of whether 
the institution had adopted CBE. These included: 

1. Federal student aid regulations 


2. Institutional business processes 


3. Costs associated with program start-up 


18 Navarre Cleary, M. (2015). Faculty and staff roles and responsibilities in the design and delivery of competency-based education programs: A C-BEN snapshot. Franklin, TN: 
Competency-Based Education Network. Retrieved from http://works. bepress.com/navarrecleary/14/. Page 9 contains discussion of new ways to conceptualize the coaching, 
advising, and mentoring roles. 

19 Bushway, D., Dodge, L., & Long, C. (2018). A leader's guide to competency-based education: From inception to implementation (p. 88). Sterling, VA: Stylus Publishing. 
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Among institutions that have existing programs or have begun building them, there were some differences 
between 2-year and 4-year institutions (see Figure 10). Of the 4-year institutions that responded, the top 
barriers to implementation included: 


1. Faculty perceptions of CBE on campus 
2. Start-up costs for CBE program 
3. Federal student aid regulations 


In contrast, 2-year institutions reported (1) institutional business processes, (2) federal student aid 
regulations, and (3) competing priority initiatives on campus as key barriers. Still, none of these 
differences were statistically significant. 


Figure 10. To what extent is the adoption of CBE at your institution helped or hindered by the 
following factors? 
IPEDS % Don’t 


Factors Level Know 
Federal Student Aid regulations and processes 4-year 42 
2-year 31 


Your institution’s business systems and 4-year 30 
processes that support CBE 2-year 18 


Other priority initiatives at the institution 4-year 29 


2-year 21 

CBE program start-up costs 4-year 44 

2-year 38 

Faculty members’ perception of CBE programs = 4-year 24 
2-year 17 

Accreditors’ regulations and processes 4-year 26 

2-year 21 


Non-federal student aid regulations and 4-year 48 
processes of the U.S. Department of Education 2-year 44 
On-campus expertise for developing CBE = 4-year 21 
programs _—- 2-year 18 

Your institution’s educational technology 4-year 21 


resources 2-year 18 

Program’s financial sustainability 4-year 44 

2-year 37 

Senior administrators’ perceptions of whether 4-year 21 
CBE is a “fit” for your institution 2-year 7 

Your ability to align industry standards to 4-year 32 
programs’ competencies 2-year 1 

The support of your institution’s leadership 4-year 23 
2-year 13 

Demand from students 4-year 49 

2-year 44 

Evidence about CBE programs’ potential to 4-year 30 
improve outcomes for students like yours 2-year 19 


Evidence about CBE programs’ potential to 4-year 43 
reduce cost for students like yours 2-year 28 


-100 -50 0 50 100 
E Significantly hindered | Somewhat hindered eal Somewhat helped es Significantly helped 
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Although many of the trends were similar for institutions that were interested in CBE but had not yet begun 
building a program, there is one notable difference: 57% cited a lack of on- campus expertise to be a key 
barrier to implementation (see Figure 11) compared with 40% of institutions with a CBE program or in the 
process of adopting a program. 


Figure 11. To what extent is interest in the adoption of CBE at your institution accelerated or slowed 
by the following factors? 

IPEDS % Don’t 
Factors Level Know 


On-campus expertise for developing 4-year 30 
CBE programs 2-year 28 


Your institution’s business systems and 4-year Al 
processes that support CBE 2-year 30 


Federal Student Aid regulations and processes 4-year 47 


2-year 32 

Faculty members’ perception of CBE programs 4-year 38 
2-year 36 

Other priority initiatives at the institution 4-year 36 
2-year 34 

CBE program start-up costs 4-year 47 

2-year 50 

Accreditors’ regulations and processes 4-year 42 

2-year 29 


Your institution’s educational 4-year 29 

technology resources 2-year 34 

Non-Federal Student Aid regulations and = 4-year 58 
processes of the U.S. Department of Education 2-year 55 
Program’s financial sustainability 4-year 58 

2-year 49 

Senior administrators’ perceptions of whether 4-year 30 
CBE is a “fit” for your institution 2-year 33 

Your ability to align industry standards to 4-year 49 
programs’ competencies 2-year 27 

The support of your institution's leadership 4-year 36 


2-year 33 
Demand from students 4-year 61 
2-year 57 


Evidence about CBE programs potential to 4-year 43 
improve outcomes fro students like yours 2-year 42 
Evidence about CBE programs’ potential to 4-year 51 
reduce cost for students like yours 2-year 44 


-100 -50 0 50 100 
BN Significantly slowed I) Somewhat slowed §) Somewhat accelerated J Significantly accelerated 
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Of those institutions that had no interest in adopting CBE (See Figure 12), federal student aid and accreditation 
regulations remained at the top of the list, but senior administrators’ perception of fit was higher than the other 
groups. It appears to be a greater barrier for 4-year institutions (74% citing it as a barrier) than for 2-year 

institutions (34%). This distinction may be important because those adopting and interested (above) cited 

senior administrators’ perception of fit to be among the top factors supporting implementation or interest. 

Ultimately, this may reinforce the notion that institutional factors, particularly those outside of academic units 
or departments, are an important part of supporting CBE implementation. 


Figure 12. Among Institutions Not Interested in CBE: Minor and Major Barriers to Interest 


Evidence about CBE programs’ potential to 2-year 
reduce students’ cost 4-year 

Ability to align competencies with 2-year 

industry standards 4-year 

U.S. ED regulations and processes (non-FSA) 2-year 
4-year 

Student demand 2-year 

4-year 

Faculty perception 2-year 

4-year4 

CBE start-up costs 2-year 

4-year 

Evidence about CBE potential to improve 2-year 
student outcome 4-year 


Business systems and processes 2-year 


4-year 
Ed tech resource 2-year 
4-year 
Leadership support 2-year 
4-year 
Program’s financial sustainability 2-year 
4-year 
On-campus expertise about CBE 2-year 
4-year 
Senior administrators’ perceptions of CBE “fit” 2-year 
4-year 
Federal Student Aid regulations and processes 2-year 
4-year 
Accreditors’ regulations and processes 2-year 
4-year 
Other priority initiatives 2-year 
4-year 
0% 20% 40% 60% 80% 100% 


|| Minor barrier ae Major barrier 
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Comparison to 2016: The broad trends are consistent with the 2016 survey findings; more than 75% of 
institutions reported that federal student aid processes and CBE program start-up costs were either a 
significant barrier or somewhat of a barrier to CBE implementation. Faculty perceptions (or skepticism) of 
CBE and resource constraints (including constraints due to competing priorities) also were among the top- 
reported barriers in 2016. 


6.Future of CBE 


Key finding: Most institutions are optimistic about the future of CBE. 


The future of CBE is likely to be influenced by the regulatory climate and the ability of schools and vendors 
to remove or bypass persistent obstacles to implementation. However, the willingness of institutions to 
expand existing CBE programs or develop new ones plays an equally important role in CBE’s growth over 
time. To better understand this, the 2018 NSPCBE survey sought to gauge expectations among responding 
institutions over whether CBE would grow nationally and at their own institution. Overall, 75% of institutions 
reported that they expect the number of CBE programs nationally to increase in the next 5 years, 24% 

expected there to be no change in the number of CBE programs nationally, and just 1% reported that they 

expect a decrease in CBE programs nationally (See Figure 13). The findings are similar across institution 

type, where 80% of 2-year institutions expect the number of programs to increase compared with 73% of 
4-year institutions. 


Responses suggest that an institution’s interest in CBE is related to its perceptions about the future of 
CBE.?° Among institutions adopting CBE or interested in doing so, expectations were relatively similar. 
Among both institutions with CBE programs or those in the process of adopting CBE, 80% expected CBE 
to increase and 20% believed the number of programs would be unchanged (see Figure 14). Furthermore, 
of institutions with CBE programs, 83% reported expecting that CBE programs at their institution would 
increase in the next 5 years. Of those institutions interested in adopting, 76% expected the number of 
programs to increase, 23% expected no change, and 1% expected the number of programs to decrease. 
However, among those institutions that were not interested in adopting CBE, 58% expected CBE to grow in 
the next 5 years, 38% expected no change, and 4% believed the number of CBE programs will decrease. 


Comparison to 2016: Data from 2016 revealed a similar set of attitudes toward the prospects for further 
CBE growth. Among institutions with existing programs, 33% reported that they expected CBE to continue to 
play a major role in their institution’s strategic plan. Among this same cohort, however, only 16% envisioned 
CBE becoming part of the higher education “mainstream.” Among schools that reported to be in the planning 
stage, only 5% anticipated mainstream potential for CBE; 32% expected CBE to useful in specialized programs 
or through the use of specific component parts. Not surprisingly, these 2016 data suggested that institutions 
with first-hand experience in operating a CBE program were more likely to anticipate further growth. Institutions 
that had yet to implement a program remained slightly less sure of continued growth. 


These data suggest that despite significant and varied barriers to CBE program growth, most institutions 

remain optimistic about the prospects for further CBE expansion. Although there is evidence that optimism 
is higher among those with direct experience in developing and managing a CBE program, the sense that 

CBE will grow going forward cuts across key categories of respondents. This positive outlook is consistent 

with the 2016 findings, and may even be interpreted as an increased sense of optimism. 


°0 These differences are statistically significant. 
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Figure 13. Expectations About the Future of CBE, by Adoption 
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Figure 14. Expectations About the Future of CBE, by Institution Level 
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CRITICAL QUESTIONS FACING THE FIELD: 
CBE IN 2018 


Findings from the 2018 NSPCBE confirm that the learner- 
centric logic for CBE remains compelling for many institutions. 
Indeed, 57% have implemented CBE or are in the process, and 
27% are interested—in sum, 84% are either implementing or 
interested in implementing CBE. A small subset of institutions 
account for the most program activity. 


A critical question is whether CBE can expand in scale and 
have an impact in traditional institutional contexts and in 
the current regulatory and policy environment. The findings 
from this survey suggest that building full CBE programs 
requires overcoming significant barriers, particularly those 
outside the purview of any individual academic unit looking 
to implement CBE, including federal financial aid regulations 
or institutionwide business processes. 


Ultimately, these barriers may explain why many institutions 
initially use only some CBE elements without adopting a full set or opt for course-level activity rather than 
full programs. However, CBE’s potential to address institutions’ learner-centric goals rests in providing 
its perceived benefits throughout a student’s experience—or for a full program rather than in one or two 
courses—and it is not yet clear whether that logic can help institutions overcome the barriers. 


Despite the current small scale of CBE and barriers to its implementation, the majority of institutions 

responding to the survey remain optimistic about the growth of CBE. This trend may reflect widespread 

acknowledgment of the commonly touted, systemic challenges to providing affordable, accessible, and 

high-quality postsecondary experiences. Or it may signal a deep reservoir of commitment to CBE as the 
best solution to these problems, perhaps leading more institutions to implement CBE in the years ahead 
despite the barriers. The NSPCBE will continue to track the level of optimism and implementation in the 

coming years to better understand that evolution, the decisions that institutions make, and the students 
they serve. 
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THE ROAD AHEAD 


When considering how to support the sustainable and 
responsible implementation of CBE programs, CBE 
stakeholders could benefit from assessing several 
critical challenges: 


& Program leaders and department chairs considering CBE 
should weigh the relative benefits and costs of starting with 
specific CBE elements versus going “all in” on implementing 
a full CBE program to understand which components will 
allow them to best navigate potential barriers. In addition, 
program leaders with operating programs can consider 
and evaluate whether their program designs indeed achieve 
the goals or benefits they intended, including serving 
nontraditional students and supporting equitable 
outcomes for all students. 


& Institutional leaders should assess whether CBE represents enough of a strategic priority that it is worth 
marshaling institutionwide resources to remove important barriers identified in this survey, including 
technology services and platforms, business processes, financial aid, and institutional research. 


= Policymakers must figure out how to balance efforts to remove regulatory barriers to innovation—as 
cited by many in this survey—with maintaining important consumer protections. The survey data do not 
provide sufficient detail to cite individual regulations nor do they provide evidence of the efficacy needed 
to help inform a path forward, but those barriers continue to be perceived by both institutions who have 
navigated the barriers and implemented a program and by those who have not yet started. 


Given the prevalence of barriers to CBE implementation, it will be critical that future research continues to 
seek a better understanding of the experience of CBE implementation and how it is serving students. Future 
years of the NSPCBE will focus on broadening the field’s understanding of where CBE is and is not being 
successfully implemented, what barriers and accelerators affect implementation, and how faculty and staff 
roles are structured. The survey also will have a deeper focus on cost to students in CBE programs, a topic 
that was not well captured by the existing questions. In addition, given early indications that these programs 
indeed serve sizable populations of older students and students of color, the survey will probe more deeply 
about how programs are designing approaches with these students in mind, including whether and how 
program leaders incorporate equity into their design and operations. With an eye toward change over time, 
the NSPCBE also will track trends in the field and question whether and how changes in the field are related 
to external factors, such as policy or regulatory changes. 
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APPENDICES 


Appendix A: Survey Descriptive Statistics 


The following tables provide details on responses to questions related to program implementation. Please 
note that these responses are based on respondents who indicated that they had a CBE program, which is 
a subset of the overall respondents. 


Table A-1. How long has your institution offered competency-based courses? 


Less than one year 6% 
1-2 years 19% 
3-4 years 26% 
5-7 years 1% 
More than 7 years 29% 
Don't know 14% 


Table A—2. How long has your institution offered entire programs that are exclusively CBE? 


Less than one year 13% 
1-2 years 12% 
3-4 years 24% 
5-7 years 10% 
More than 7 years 11% 
Don't know 30% 


Table A-3. Do your CBE programs... 


Don’t Know | No, none do | Yes, some do | Yes, all do 
Lead to a certificate, undergraduate, or graduate 
; 8 8 7% 10% 24% 60% 
degree, if completed? 
Require mastery learning of ALL competencies 
oe y g 11% 11% 17% 61% 
in a program? 
Primarily require students to demonstrate their 
; ; 10% 0% 36% 54% 
competency via authentic assessments? 
Use "backward design," where the competencies 
g e 26% 10% 27% 37% 


to be mastered drive students’ learning journey? 


Table A—4. Can students access federal financial aid to pay for your CBE offerings? 


Yes 55% 
No 22% 
Don't know 24% 
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Table A-5. For federal financial aid purposes, our institution... 


Maps competencies to credit hours to award financial aid 68% 
Has been approved for "direct assessment" by the 17% 
US Department of Education 

Other 15% 


Table A-6. At which award levels are your undergraduate CBE programs offered? (Check all that apply.) 


Non-credit 14% 
Certificate 51% 
Associate’s degree 38% 
Bachelor's degree 53% 


Table A-7. In what disciplines are your undergraduate CBE programs offered? (Check all that apply.) 


Biological and Life Sciences 1% 

Business Administration 40% 
Computer and Information Sciences and Support Services 47% 
Construction Trades 13% 
Education 17% 
Liberal Arts and Humanities 10% 
Mechanic and Repair Technologies 12% 
Nursing and Health Professions 33% 
Physical Sciences (e.g., Chemistry, Engineering) 3% 

Social Sciences (e.g., Psychology, Sociology, Political 38% 
Science, Economics) 

Other 19% 


Table A-8. For the most recent academic year for which you have data available, about how many 


undergraduate students: 


500- More 
0-50 51-100 | 101-200 | 201-499 1000 than 1000 
Are enrolled in CBE programs that are entirel 
pee : 53% 13% 9% 11% 9% 6% 
competency-based? 
Are expected to be enrolled in your CBE programs 
: rs 13% 18% 20% 21% 19% 9% 
that are entirely competency-based within 5 years? 
Have graduated from CBE programs that are 
g Prog 62% 12% 1% 11% 3% 4% 


entirely competency-based? 
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Table A-9. For the most recent data you have available, which best describes the composition 
of undergraduate students enrolled in your certificate and degree programs that are entirely 


Percent of undergraduate students enrolled in your certificate 
or degree programs that are entirely competency-based 


competency based? 


0-24% 25-49% 50-74% 75% or more 
Percent who are white, non-Hispanic 33% 21% 40% 6% 
Percent who are at least 25 years old 17% 12% 28% 44% 
Percent who had college credit at the time of admission 41% 11% 11% 37% 
Percent who are veterans or active duty military 96% 4% 0% 0% 


Note. Fewer than 50 institutions reported data for this item. 


Table A-10. At which award levels are your graduate (post-baccalaureate) CBE programs offered? 
(Check all that apply.) 


Non-credit 0% 
Certificate 43% 
Master's degree 52% 
Professional degree 12% 
5% 


Doctoral degree 


Table A-11. In what disciplines are your graduate (post-baccalaureate) CBE programs offered? 


(Check all that apply.) 

Biological and Life Sciences 0% 

Business Administration 23% 
Computer and Information Sciences and Support Services 21% 
Construction Trades 0% 

Education 28% 
Liberal Arts and Humanities 2% 
Mechanic and Repair Technologies 0% 
Nursing and Health Professions 36% 
Physical Sciences (e.g., Chemistry, Engineering) 0% 
Social Sciences (e.g., Psychology, Sociology, Political 1% 
Science, Economics) 

Other 5% 


Note. Fewer than 50 institutions reported data for this item. 


SS (A | D5 


STATE OF THE FIELD | FINDINGS FROM THE 2018 NATIONAL SURVEY OF POSTSECONDARY COMPETENCY-BASED EDUCATION (NSPCBE) 


Table A-12. For the most recent academic year for which you have data available, about how many 
graduate (post-baccalaureate) students are enrolled in CBE programs that are entirely competency-based? 


0-50 61% 
51-100 29% 
101-200 0% 
201-499 0% 
500-1000 2% 

8% 


More than 1000 
Note. Fewer than 50 institutions reported data for this item. 


Table A-13. Five years from now, about how many graduate (post-baccalaureate) students do you 
expect to be enrolled in your CBE programs that are entirely competency-based in each year? 


0-50 31% 
51-100 33% 
101-200 18% 
201-499 2% 
500-1000 4% 

11% 


More than 1000 


Note. Fewer than 50 institutions reported data for this item. 


Table A-14. For the most recent data you have available, about how many graduate (post-baccalaureate) 
students have ever graduated from CBE programs that are entirely competency-based in total? 


0-50 60% 
51-100 14% 
101-200 0% 
201-499 14% 
500-1000 1% 

4% 


More than 1000 
Note. Fewer than 50 institutions reported data for this item. 


Table A-15. For the most recent academic year for which you have data available, which best describes 
the composition of graduate students enrolled in your certificate and degree programs that are entirely 


competency based? 
Percent of graduate students enrolled in your certificate 
or degree programs that are entirely competency-based 
75% or more 


Don’t Know 0-24% 25-49% 50-74% 
Percent who are white, non-Hispanic 26% 2% 19% 47% 1% 
Percent who are at least 25 years old 26% 0% 16% 21% 37% 
Percent who had college credit at the time 
Cine 8 12% 23% 0% 16% 48% 
Percent who are veterans or active 
duty rnilttary 52% 44% 4% 0% 0% 


Note. Fewer than 50 institutions reported data for this item. 
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Table A-16. To what extent does a senior administrator lead efforts related 
to competency-based approaches? 


Not at all 16% 
A little 11% 
Mostly 21% 
To a great extent 44% 
Don’t know 8% 


Table A-17. To what extent is CBE incorporated into your institution’s strategic plan? 


CBE is not in our current strategic plan and isn't likely to 


3 8% 
appear in a future one. 
CBE is not in our current strategic plan but is likely to play 17% 
a minor role in a future one. 
CBE is not in our current strategic plan but is likely to play 3% 
a major role in a future one. 
CBE is in our current strategic plan in a minor way. 40% 
CBE is in our current strategic plan in a major way. 16% 
Don't know 11% 


Table A-18. To what extent is funding for CBE available for each of the following? 


No funding Some funding Major funding 
available for CBE available for CBE available for CBE 
Departments at your institution 34% 53% 14% 


Your institution 45% 41% 14% 


Table A-19. How do your students access and participate in CBE offerings? 


Our CBE courses are accessed entirely online. 37% 
Our CBE courses are accessed in a hybrid or blended modality, combining online 30% 
and face-to-face interactions. 

Our CBE courses are predominately face-to-face, with a few online assignments. 17% 
Our CBE courses are entirely face-to-face. 16% 
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Table A-20. For each of the following CBE characteristics, consider the current CBE offerings which you 
are most familiar. How often, or to what extent, are... 


Don’t Occasionally/to Frequently/to 
know Never some extent a great extent Always 
Curricula clearly organized around 
ae 0% 0% 6% 35% 59% 
measurable competencies. 
Competencies aligned to specific 
‘ tee ae 6% 0% 15% 41% 38% 
workforce opportunities. 
Offerings designed to be completed 
- : : : P T% 2% 22% 17% 52% 
over variable periods of time. 
Students proceeding through content 
eu eras 6% 2% 22% 16% 54% 
and assignments at a flexible pace. 
Students receiving real-time, personalized 
g is 6% 1% 11% 48% 34% 


assessments of their learning progress. 

Students’ pathways through courses 

personalized based on assessments 12% 9% 23% 23% 33% 
of their learning progress. 


Table A-21. Generally, how does the level of effort required of faculty and support staff in CBE offerings 
compare to non-CBE offerings at your institution? 


CBE requires much more work than non-CBE 22% 
CBE requires somewhat more work than non-CBE 38% 
Roughly the same level of work than non-CBE 22% 
CBE requires somewhat less work than non-CBE 16% 
CBE requires much less work than non-CBE 0% 
Don’t know 2% 


Table A-22. Generally, how does each of the following differ in your institution’s CBE offerings compared 
to non-CBE offerings? 


Too early Significantly | Slightly worse Slightly better | Significantly 
to judge or | worseinCBE | in CBE than Roughly in CBE than | better in CBE 
don’t know | than non-CBE non-CBE the same non-CBE than non-CBE 

seems 38% 0% 2% 41% 16% 2% 
communication 

Learning outcomes 43% 0% 0% 26% 20% 10% 
Completion rates 43% 0% 6% 36% 4% 10% 
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Table A-23. Please identify how your institution supports CBE graduates. My institution... 


Expedites the transfer of credits to and from other institutions. 12% 
Matches graduates with employers based on student performance and mastery of specific competencies. 8% 

Connects graduates to a broader community of CBE alumni. 10% 
Identifies areas of greatest workforce needs and opportunities. 32% 
Provides ongoing opportunities for graduates to continue augmenting their education through new CBE 10% 
courses, certificates and degrees. 

Other 8% 

Has no current CBE graduates. 44% 
Don’t know 2% 


Note. Fewer than 50 institutions reported data for this item. 


Table A-24. Which services do you outsource to an external vendor to support your institution's 


CBE offerings? 

Learning Management System (LMS) 66% 

Instructional design 2% 

Client relationship management (CRM) 16% 
Marketing and recruitment 2% 
Enrollment management 2% 
Development of assessments or competencies 0% 
Development of online or blended CBE courseware 4% 

Learning portfolio services 29% 

Other 2% 

Don't know 16% 

Note. Fewer than 50 institutions reported data for this item. 

Table A-25. How is CBE course content created? 

Developed largely from scratch by our faculty. 90% 
Bundled as part of our learning management system (LMS) or similar tool. 6% 
Developed by a third-party specialist. 0% 
Developed by a combination of faculty and third-party specialists. 10% 
Used, adapted, or modified content from outside the university. 8% 
Used, adapted, or modified content from within the university. 18% 
Developed in conjunction with specific employers or industry associations. 38% 
Other 4% 
Note. Fewer than 50 institutions reported data for this item. 

Table A-26. How are CBE competencies created? 
Developed largely from scratch by our faculty. 92% 
Developed by a third-party specialist. 0% 
Developed by a combination of faculty and third-party specialists. 10% 
Developed in conjunction with specific employers or industry associations. 31% 
Other 2% 


Note. Fewer than 50 institutions reported data for this item. 
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Table A-27. Have you used the following resources as you developed your program? 
(For institutions with programs or in the process of adopting) 


Quality Framework published by the Competency-Based Education Network (CBEN) 28% 
The Connecting Credentials Framework/Beta Credentials Framework 11% 
LEAP/VALUE rubrics published by AAC&U 30% 
Resources provided by the U.S. Department of Labor (O*NET, Building Blocks) 36% 
Degree Qualifications Profile 22% 
Employer or industry competency models 60% 


Table A-28. Have you heard of any of the following resources that can be used to support CBE program 
development? (For institutions with interest, but no program) 


Quality Framework published by the Competency-Based Education Network (CBEN) 29% 
The Connecting Credentials Framework/Beta Credentials Framework 0% 
LEAP/VALUE rubrics published by AAC&U 71% 
Resources provided by the U.S. Department of Labor (O*NET, Building Blocks) 0% 
Degree Qualifications Profile 43% 
Employer or industry competency models 43% 


Note. Fewer than 50 institutions reported data for this item. 
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Appendix B: Technical Documentation 


This appendix outlines the methods of the National Survey of Postsecondary Competency-Based Education 
(NSPCBE), sponsored by Lumina Foundation and conducted by the American Institutes for Research (AIR). 
The Web-based survey was administered in the summer of 2018, from June 11 to July 27. 


POPULATION AND SAMPLING 


The NSPCBE was intended to be administered to administrators at all 4,291 2- and 4-year institutions of 
higher education in the United States. The list of institutions was drawn from the Integrated Postsecondary 
Education Data System (IPEDS). Because this is a census, no sampling occurred. Not all institutions 
were contacted for the survey, though; if the institution could not be successfully “rostered”—meaning 
that the research team could not obtain contact information for at least one administrator who may be 
knowledgeable about CBE programs—it was not contacted. 


In total, 4,668 provosts, institutional research directors, and other administrators representing 3,043 

institutions were sent an invitation to participate on June 12, 2018. Contact information was obtained 
from several sources, including directory files purchased by the research team and contact information 
reported in the previous Eduventures survey. 


Because this was a census and not a probability sample, no estimates of sampling error will be reported. 


RECRUITMENT AND SURVEY FOLLOW-UP 


The surveys were administered via the Web with e-mail notifications and reminders. Prior to launch on June 
11, AIR and Eduventures notified their partners by direct e-mail and announcements on various electronic 
mailing lists. During the field period, AIR and Eduventures contacted potential survey respondents through 
e-mail reminders and direct mail, supplemented by presentations and announcements about the survey 
through advisory board member outlets. 


RESPONSE RATES 


The overall response rate for this survey was 16%; 501 of the 3,043 rostered institutions responded. 
Priority institutions—identified based on their past interest in CBE or the size and makeup of their student 
population—had a response rate of 23%. 


Response rates may be calculated in a variety of different ways. The American Association for Public 
Opinion Research (AAPOR) has standardized response rate calculations across the survey and polling 
industry, providing a variety of different options for researchers.” In this study, AAPOR’s Response Rate 2 
(RR2) was used to calculate response rates: 


AAPOR RR2 = (Completes + Partials) /(Completes + Partials + Eligible Nonrespondents) 


Partial responses were counted as such if the respondent completed the screener (up to question 5; see 
Appendix A) but did not complete the rest of the survey. If individuals logged into the survey but did not 
complete the screener, they were considered nonrespondents. Individuals who completed all relevant survey 
items were counted as completers. 


21 https://www.aapor.org/Standards-Ethics/Standard-Definitions-(1).aspx 
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Institution-level response rates. The research team targeted multiple individuals at each institution 
and, at 24 institutions, two individuals responded. Analyses included only a single respondent for each 
institution, based on a series of rules as follows to identify and keep the most knowledgeable respondent 
at the institution: 


= Completeness of data: the respondent with the most complete data took precedence. 


H Scope of knowledge: if both respondents had the same number of questions completed, the respondent 
who indicated “institutionwide” knowledge of CBE approaches in question 2 of the survey took precedence. 


™ Role: if the response to question 2 was missing or if both individuals selected that option, question 1 
was used to identify the person most likely to be knowledgeable about activity across the institution. 
Vice provosts took precedence, followed in order by deans, provosts, institutional review directors, 
department chairs, presidents, and faculty. 


In addition to calculating the response rate at the individual level, AIR and Eduventures also calculated 
the response rate at the institution level, again using AAPOR RR2. In this response rate calculation, the 
numerator includes institutions with at least one complete or partial response, per the definitions described 
above. The denominator was the 3,043 institutions for which valid contact information was available. 


WEIGHTING 


Broadly, the target population for the NSPCBE consists of institutions of higher education in the United 
States. For weighting purposes, the target population was defined as institutions meeting both of the 
following criteria: 


—™ The institution is listed in IPEDS??; and 


™ The institution was successfully “rostered,” meaning that the project team was able to obtain contact 
information for at least one administrator who may be knowledgeable about CBE programs. 


This corresponds to the definition used in the calculation of response rates. Note that this definition 
implies that institutions that are listed in IPEDS but were not rostered are treated as ineligible for the 
survey. Thus, weighted estimates will be representative only of rostered institutions; they will not be 
representative of all institutions in the United States. 


To calculate the weights, the universe of institutions was first partitioned into adjustment cells using a 
classification and regression tree (CART).2° CART is a machine learning algorithm that automatically identifies 
predictors that are associated with a dependent variable of interest—in this case, IPEDS variables associated 
with the likelihood of being a respondent. The algorithm then successively partitions the universe into cells 
defined by those variables, with the aim of maximizing between-cell variability in the response rate. 


For all institutions i assigned to a given cell b, the weight was calculated as: 


(1) w.=U, 
R, 


22 One exception was the College of Traditional Midwifery. This institution was not in IPEDS but was included because it is part of the CBEN network and is known to CBE users. 
23 The specific CART implementation was the rpart function in R, available in the rpart package. A minimum cell size of 35 and a complexity parameter of 0 were specified. 
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where U, is the universe size in cell b and R, is the number of contacts in cell 6. Equation 1 is the equivalent 
of the inverse of the response rate within cell b. 


The following IPEDS variables were selected by the CART analysis to define the nonresponse adjustment cells: 
m STRATUM 

M C15BASIC (Carnegie classification 2015) 

™ CCBASIC (Carnegie classification 2005/2010) 

™ STABBR (state) 

™ EFTOTLM (number of enrolled men, categorized into deciles) 

™ EF2MORT (number enrolled with two or more races, categorized into deciles) 

m EFYASIAW (number of enrolled Asian women, categorized into deciles) 


Across the 501 respondents, the final weights, by construction, sum to the universe size of 3,043 institutions. 
The final weight is undefined for the remaining 2,541 institutions in the universe that were not respondents. 
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